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Abstract 



The numerous reasons for teaching thinking have been tied to the perceived needs of the 
present and future technological society. Expectations are placed on individuals to be 
self-sufficient, responsible citizens, and part of an efficient work force, oriented 
toward an information age. Endorsement of this objective and attempts to teach thinking 
have resulted in the proliferation of many learning/thinking programs. Few studies to 
date have, however, systematically and tongitudinally evaluated any of the available 
programs to determine which ones are most effective and most easily integrated into the 
regular school curriculum. More importantly, there is little empirical data comparing 
the efficacy of the programs at different grade levels or with different types of learners. 

The purpose of the Cognitive Education Project, centered at the University of 
Alberta, was to undertake a three-year longitudinal evaluation of two cognitive education 
programs. The critical difference between the two experimental programs was that one 
(Feuerstein's Instrumental Enrichment - I.E.) was taught out of curricular content, 
while the second program (Strategies Program for Effective Learning/Thinking - 
S.P.E.LT.) was taught directly within curricular content. 

Specifically, the effectiveness of the I.E. and S.P.E.LT. cognitive education 
programs was compared with traditional instruction at two initial grade levels (grades 4 
and 7) for three diagnostic groups (gifted, learning disabled and normal achievers). The 
comparison was done in terms of: 

a. the effects of the programs on students* affect and motivation, academic 
achievement, cognitive ability, and learning/thinking and problem solving 
strategies; 

b. the differential impact of the programs; 

c. the feasibility of implementing learning/thinking strategies instructional 
programs as part of the regular curriculum of schools; and 

d. identifying appropriate methods for providing the level and quality of 
teacher training necessary for implementation. 

These general objectives gave rise to a number of questions spanning student, 
teacher, parent and administrator responses. The study was implemented in two phases 
starting in 1984 and 1985 respectively and overlapping, with phase 1 ending in 1987 
and phase 2, in 1988. It utilized a repeated measures factorial design involving three 
types of instructional programs, three categories of students, and two initial grade 
levels (grades 4 and 7). The complete study provided four data points. 
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In the course of the implementation, teachers of the control condition taught as 
usual, whereas teachers assigned to the two cognitive education procedures received 
intensive inservice training prior to classroom strategy instruction. 

In identifying subjects for the study, intellectual, academic and behavioral 
characteristics were used, resulting in the selection of 900 students from an initial 
population of 4.000. Based upon intelligence test scores and achievement three groups 
of subjects were identified as gifted, average and learning disabled. 

Apart from obtaining responses from students, questionnaires were administered 
to teachers, parents and administrators to assess their perceptions of the different 
programs. The results of the study indicate that: 

1. Cognitive education was effective in improving student thinking, especially 
for the grade 4 learning disabled, and to a lesser extent the gifted, in reading 
comprehension, and comprehension monitoring sl<ills. Students' strategic behavior 
generally improved across grade and diagnostic groups. 

2. Though I.E. was effective, S.P.E.LT. tended to produce more positive changes 
in students' overall performance. 

3. Teachers', parents', and administrators' responses were positive towards the 
two cognitive education programs, with more favorable remarks being made for 
S.P.E.LT. The teachers involved in the study indicated general appropriateness of both 
experimental programs especially for grade 4 students. The vast majority of 
experimental teachers said the inservices provided were appropriate and that they would 
continue to use the instructfonal procedures from their respective program. As well, 
the teachers said they would recommend the two programs to their colleagues. Parents 
also indicated observation of positive changes in their youngster's self-confidence, task 
persistence, accepting alternative points of view, originality of thinking and ques- 
tioning, etc. 

4. The question of whether either of the cognitive education programs was more 
effective for specific groups of children did not receive a definitive answer and may have 
to be further examined. On the whole, the experimental programs appeared to be most 
effective for learning disabled students, and to a lesser extent, the gifted at both grade 
levels. 
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The Cognitive Education Project 
Summary Report 



The purpose of this report is to summarize the major features and findings of the 
Cognitive Education Project. First, the rationaie, objectives and research questions of 
the project are outlined. These are foliowed by a brief description of the two cognitive 
education programs (Instrumental Enrichment and Strategies Program for Effective 
Learning/Thinking) selected for comparison in the project. Research methodology and 
findings are then presented. Finally, conclusions and recommendations are specified. 



Rationale 

Over the past decade, there has been a significant increase in interest and desire from 
teachers, school districts, colleges, and universities to teach thlnkinc} skills. This 
interest has been associated with the emergence of various cognitive education programs 
aimed at enhancing students' cognitive and metacognitive skills. This has been done with 
the hope that students might become more independent learners and more efficient 
problem solvers. 

Although cognitive education programs have been growing in number and 
popularity, many questions concerning their effectiveness remain. Part of the reason is 
the lack of long-term evaluation of these programs. First, the packages available today 
tend to vary in terms of scope, skill development, age/grade suitability, training 
requirements, curriculum integration, cost, and instructional methodology. Few studies 
to date have systematically evaluated any of the available programs to determine which 
ones are most effective and most easily integrated within the regular school curriculum. 
More importantly, there is little empirical work comparing the efficacy of the progra.Tis 
at different grade levels for different types of learners. The results of these studies are 
often equivocal with regard to program effectiveness. 

Associated with this lack of comprehensive evaluation is the question of selecting 
from alternative approaches to teach thinking. That is, linking this evaluation issue to 
instruction, the central question becomes how to teach learning/thinking. Educators are 
faced with choosing between an in-content or out-of-content instructional 
approach. Which would work better for students? Would there be differences if we 
compare these two approaches at different grade levels for different types of learners? 
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Basing our condustons and subsequent action on subjective preferences alone would be 
inadequate. The seeming course is to conduct an evaluation study: putting in-content and 
out-of-content programs in place in selected schools and evaluating the results. With 
this main purpose, the Cognitive Education Project (CEP) was inaugurated in the fall of 
1984. 



Objectives and Research Questions 

The Cognitive Education Project wss a cooperative venture involving: 1) Alberta 
Education, Govemment of Alberta; 2) the Department of Educational Psychology, the 
University of Alberta; and 3) various school jurisdictions in north-centra! Alberta. It 
was established with the general purpose of undertaking a long-term evaluation of two 
cognitive education programs (out--of-content versus in-content) in relation to 
traditional instruction in elementary and junior high classrooms. 

For the out-of-content approach, Feuerstein's Instrumental Enrichment (I.E.) 
was selected because it is one of the most comprehensive and field-tested learning and 
thinking programs available to date. It represents an out-of-content instructional 
approach utilizing pa^er-pencii tasks and intensive teacher-pupil discussion to teach 
learning/thinking skills. In contrast, the Strategies Program for Effective 
Learning/Thinking (S.P.E.LT.) was chosen as an in-content instructional approach. It 
integrates the features of several prominent cognitive theorists and intervention 
procedures, and it emphasizes the teaching of learning/thinking strategies directly 
within content across the curriculum. 

The effectiveness of cognitive education — represented by the I.E. and S.P.E.LT. 
programs — was compared with the effects of traditional instruction at two initial grade 
levels (grades 4 and 7) for three diagnostic groups (gifted, learning disabled and 
average achievers). Specifically, the objectives of the project were fourfold: 

1 . to assess the relative effectiveness of the two programs in terms of their impact on 
students* affect and motivation, academic achievement, cognitive ability, and 
learning/thinking and problem-solving strategies; 

2. to examine the differential effects of the programs on gifted, normal achieving, and 
learning disabled students; 

3. to ascertain the feasibility of implementing learning/thinking strategies 
instructional programs on a large scale as part of the regular curriculum of 
schools; and 
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4. to identify appropriate methods for providing the level and quality of teacher 
training necessary for implementation. 

In response to the objectives listed above, nine research questions were formulated. 

1 . What are the relative effects of the different cognitive education programs compared 
to traditional instruction on gifted, learning disabled, and average achievers with 
respect to the following dimensions? 

a perceived competence 

b. perceived locus of control 

c. performance in reading comprehension 

d. performance in arithmetic problem solving 

e. use of specific strategies employed in solving problems 

2. Is/are the training program(s) more appropriate at different ages for different 
diagnostic groups? 

3. Do the pupils continue to maintain and/or alter their level of performance following 
the withdrawal of training? 

4. What is the nature of strategy monitoring for each of the three groups across the 
different age/grade groupings? To what extent can the cognitive education programs 
be implemented as intended by program developers? 

5 What is the nature of the strategies utilized by each of the groups across the 
different age/grade groupings prior to instruction and at the conclusion of the 
study? 

6. What are parents*, teachers*, and administrators* opinions regarding the cognitive 
education programs? 

?• What are the teachers* and administrators* opinions regarding inservice and 
consultative assistance provided for cognitive education programs? 

8. What guidelines for preservice and inservice programs for teachers seem 
appropriate? 

9. How well are the programs implemented by teachers of cognitive education? Do 
teachers learn and implement the cognitive education strategies? How appropriate 
and effective are the inservice and consultation provided? 
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The Two Selected Programs ^ Instrumental Enrichment and 
Strategies Program for Effective Learning/Thinking 

To clarify the characteristics of experimental treatments in the research methodology, 
the two cognitive education programs selected for comparison in the study are briefly 
described here. 

Instrumental Enrichment 

Feuerstein's Instrumental Enrichment (I.E.) (Feuerstein, Rand, Hoffman, & Miller, 
1980) was selected to represent the out-of-content approach to instruction. This 
program was originally designed for culturally disadvantaged children and youth. 
However, it is currently being used with a broader population of children in upper 
elementary, junior, and senior high schools. A distinguishing feature of I.E. is its 
emphasis on the importance of mediation for strategy development. Strategies are ways 
to facilitate the acquisition, manipulation, integration, storage and retrieval of 
information across situations and settings. For Feuerstein, training for such strategy 
development is through a **mediated learning experience". This means that an adult 
mediator elicits behaviors from the child that lead to the solving of the problem, and 
then the child comes to understand the goals and strategies of the task. Thus in I.E., 
social interaction is important, because it is believed that it is not the content, but the 
means of interacting that is internalized by the child. 

The Feuerstein program utilizes pencil-and-paper tasks with related intensive 
teacher-student discussion. It consists of 15 'instruments' or pencil-and-paper 
exercises as follows: Organization of Dots; Analytic Perception; Orientation in Space I, 
II. Ill; Comparisons; Categorization; Instructions; Family Relations; Illustrations; 
Numerical Progressions; Temporal Relations; Stencil Design; Transitive Relatfons; and, 
Syltogisms. While the names of the individual instruments indicate the dimenstons of 
the program, the program is intended to be content-free. The term content-free is 
intended to convey the idea that the contents of any particular exercise are merely a 
vehicle, or an instrument, to achieve the overall goals of the program. The major goal of 
I.E. Is to enhance the cognitive modifiability, that is, learning potential, of the 
individual. 

The I.E. program can be integrated into the regular school curriculum, but is 
taught in a decontextualized form first. Typically, I.E. instruction extends over a two- 
to three-year period, with a minimum of three sessions per week devoted to work on the 
instruments. An I.E. lesson normally begins with a ten-minute introduction, followed by 



ERLC 



15 



5 



individual work, class discussion and summary. The teacher ensures that adequate 
mediational experiences are provided to students especially in the introductory and 
discussion stages. A typical lesson in I.E. appears to be highly perceptual with strong 
visual-motor factors; however, in reality, the product resulting from, say, the 
connecting of dots to form a pattern in the Organization of Dots instrument, is a very 
minor part of I.E. The program is characterized by students' discovering a pattern in the 
instruments through mediation; determining the underlying principle, then "bridging** 
this principle to other examples. It is this dynamic invotverr )\ of the teacher in a 
dialogue with the student, riong with the change in orientation from product to process, 
that depicts this program. 

Strategies Program for Effective Learning/Thinking 
The Strategies Program for Effective Learning/Thinking (S.P.E.L.T.) (Mulcahy, Marfo, 
& Peat, 1984) was selected as the in-content program in the study. It was initially 
aimed at children in the upper elementary and junior high school grades. It has since 
been extended to high school and college populations. Similar to the I.E. program, the 
S.P.E.LT. approach also aims to foster strategic learning and emphasizes the importance 
of the interactbns between the teacher and students in the process of strategy 
development. In the S.P.E.LT. program, cognitive strategies are conceptualized as 
internally organized skills or control processes by which learners regulate their 
cognitive and/or learning behavior. A learner's repertoire of strategies is thus seen as a 
set of tools that enables him/her to more effectively and efficiently activate and regulate 
important cognitive activities such as attention, compr6hensk>nt retention and retrieval 
of inf6rmatk)n, thinking, and problem solving. The teacher plays the role of a mediator 
between the learner and the external world, structuring the learning environment and 
providing opportunities necessary to establish and improve strategic behavior in 
learning^ thinking, and problem-solving situations. A general teaching orientation is 
embedded within S.P.E.L.T. whereby the teacher's goal in all planning and instruction is 
to actively involve the student in the learning process (Peat, Mulcahy, & Darko-Yeboah, 
1989). 

S.P.E.LT. however, differs from I.E. in its instructional context. While I.E. has 
been designed as a stmctured package to be taught independently of existing curriculum 
content, the S.P.E.LT. approach hokis that the teaching of learning/thinking strategies 
should take place within content and not as an independent or isolated curricular 
activity. With regular school curriculum as a vehicle, S.P.E.LT. through inservice 
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demonstrates how the teacher can use specific strategies to activate and regulate 
students* learning activities. 

Moreover, S.P.E.LT. utilizes a three-phase instructional perspective. It 
progresses from the first phase — direct teaching of strategies in content areas — to the 
second phase — teaching for strategy transfer — and finally, to the third phase where 
students can themselves generate new strategies to acquire, analyze, and apply 
information and ideas. The active involvement of the students in the learning process is 
maintained as a goal throughout the three phases of S.P.E.LT. Instruction. For both 
Phases II and III, the teacher engages In Socratic Dialogue — an interactive relationship 
between the teacher and students, where the teacher leads the students through 
questioning to discover relationships for themselves. Thus, mediational teaching is also 
a feature of the S.P.E.LT. program. 

In sum, the common goal of the two programs is to help students learn 'how to 
learn' and thus become independent, organized, active, and purposeful thinkers and 
problem solvers. The critical factor for distinguishing between the two is the nature of 
integration of the program into the curriculum. I.E. is considered a detached program 
because it Is first taught without using curriculum content and later integrated into the 
curriculum. On the other hand, S.P.E.LT. is described as an em^^ecftfecf program because it 
is taught directly using curriculum content. 

To evaluate the effectiveness of the out-of-content approach of I.E. versus the in- 
content approach of S.P.E.LT. in relation to conventional instruction, the following 
research methodology was developed. 



Methodology 

Research design 

The study was a three-year longitudinal evaluation study implemented in two phases 
(i.e., phase 1: 1984 - 87; and phase 2: 1985 - 88)^. It utilized a repeated measures 
factorial design involving the three types of instructional programs (I.E., S.P.E.L.T., and 
Control), three categories of students (gifted, average, and learning disabled), and two 



^ Due to budget restraints the study was conducted in two major phases. Phase I (1984 - 

1987) began in October 1984 and ended in June 1987. Phase II (1985 - 1988) started 
In October 1985 and ended in June 1988. Thus, for each phase, it was a three-year 
period of evaluation. Nevertheless, for the whole study, it took four years (1984 - 

1988) to complete. 
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initial grade levels (grade 4 and grade 7). The complete study involved four data points 
(repeated measures): pre-test in xhe fall of the initial year, and two post-tests in 
succeeding May/June periods corresponding to the end of grades 4 , 5, 7, and 8, and a 
maintenance post-test at the end of grades 6 and 9 (see Table 1). 

Table 1 
Overall Study Design 



Oct. 84 June 85 Oct. 85 June 86 June 87 June 88 

Phase 1: 

Grade 4 

Air PPE PT1 _ PT2 IVIaintenance 

I.E. Pl^ PT1 _ PT2 Maintenance 

COfTFIOL PRE PT1 PT2 Maintenance 



Phase 2: 

Grade 4 

I.E. PRE PT1 PT2 Maintenance 

S.P.E.LT PRE PT1 PT2 Maintenance 

Air PRE PT1 PT2 Maintenance 

• Ail refers to all 3 instructional conditions. 
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Program implementation 

Teachers assigned to the control condition (traditional instruction) were told to teach as 
usual, whereas teachers assigned to the two cognitive education procedures received 
inservice training from project staff prior to giving strategy Instruction, Thus students 
In the control condition received traditional instruction, while students involved in the 
two cognitive education programs received a minimum of 120 minutes of strategy 
instruction per week over two school years. Strategy instruction was followed by one 
year of maintenance, during which all strategy instruction was withdrawn. Since I.E. is 
an out-of-content program, teachers were required to take time out of a variety of 
curricular content areas to implement the program. Essentially, the I.E. instruction 
time was taken from language arts. For S.P.E.L.T., teachers incorporated strategy 
instruction across content areas, and language arts was the major content medium for 
the S.P.E.LT. instruction. 

Subject identification 

Based on intellectual, academic and behavioral characteristics, about 900 pupils 
comprising gifted, average, and learning disabled students were identified from the total 
initial population of 2,400 students in 1984-85 and 1,600 students in 1985-86. 
Selection criteria were as follows: 

The gifted students selected for this project were those who: 

• obtained scores of 115 or higher on the verbal and the non-verbal sub-scales 
of the Canadian Cognitive Abilities Test (CCAT), 

• were rated as being above average in achievement in reading and at/or above 
grade level in math on the Canadian Achievement Test (CAT), 

• were rated as being above the mean (of the total study population) on all 
three of the Renzulli and Hartman Scales for *he Rating of Behavioral 
Characteristics of Superior Students (SRBCSS) categories (i.e., motivation, 
learning and creativity characteristics). 

The average-achieving students included in this study were these who: 

• obtained scores within one standard deviation of the mean on both the verbal 
and nonverbal sub-scales of the CCAT (85-115), 

• obtained achievement scores on the reading and math sub-scales of the CAT 
within approximately one standard deviation of the mean. 

The learning disabled students identified in this study were those who: 

obtained scores within one standard deviation of the mean on both the verbal 
and nonverbal sub-scales of the CCAT (85 - 115), 
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• obtained achievement scores of approximately one standard deviation or more 
below the mean on the reading sub-scale of the CAT. 
It can be seen that in the subject identification process, reading was chosen as the 
major academic measure because it is one of the most important skills necessary for 
school success. Furthermore, the majority of learr^ing disabled children experience 
learning difficulties in this area. Consequently, it was the critical achievement measure 
used in the identification of all three diagnostic groups (i.e., gifted, average and learning 
disabled). 

Assessment of program effects 

The instruments utilized in this study can be grouped into four categories: (1) cognitive 
ability, (2) academic achievement, (3) affective perceptions, and (4) cognitive 
strategies. 

To assess general intellectual/cognitive ability, the Canadian Cognitive 
Abilities Test (CCAT) was administered. This test measures verbal, quantitative and 
nonverbal reasoning abilities. 

For academic achievement, the Canadian Achievement Test (CAT) was 
utilized. This test consists of two separate batteries measuring skills in reading 
(vocabulary, comprehension) and mathematics (computation, concepts, application). 

To assess affective perceptions, several measures were employed: 

a. Perceived competence: Harter's Perceived Competence Scale was used to 
measure students' self-perceptions in four areas (i.e., cognitive, social, 
physical and general). 

b. Self-concept Coopersmith's Self Esteem Inventories were employed to 
assess evaluative attitudes toward the self in social, academic, family and 
personal areas of experience. 

c. Locus of control: CrandalPs Intellectual Achievement Responsibility 
Questionnaire (lARQ) was administered to examine students* beliefs 
regarding responsibility for outcomes in academic achievement 
situations. 

To measure cognitive strategies, several tests were selected or developed: 

a. Reading awareness: Paris' Reading Awareness Questionnaire was utilized 
to examine students* awareness of the evaluation, planning, and regulation 
skills involved in reading. 

b. Reading strategies: A ck)ze task was developed to evaluate students' reading 
comprehension abilities and strategies. It is a procedure in which words 
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are systematically omitted and students are required to fill in the blanks 
with the appropriate words. 
0. Comprehension monitoring: An error detection task was designed to assess 
students* comprehension monitoring skills. This task requires students to 
detect anomabus information in the passages; their awareness of faulty 
comprehension is taken as a measure of their compiehension nx)nitoring. 

d. Perceived problem-solving ability: Heppner and Petersen's Problem- 
Solving Inventory (PSI) was adopted to examine the underlying 
dimensions of students' perceptions of their real-life, personal problem- 
solving process. 

e. Problem solving strategies: A problem-solving task was developed to 
assess students' problem-solving strategies in mathematics. This task 
requires students to think aloud (verbalize their thoughts) as they try to 
solve the given math problems. 

To test the effects of the experimental programs, most of the above measures 
were imp'^^mented in both the pre-test and post-test phases. Measures of cognitive 
strategies, however, were not available at the pre-test point and were thus 
administered at the post-test point only. 

Assessment of participants' perceptions 

In addition to the above criterion measures, participants* perceptions with respect to 
their involvement in the Cognitive Education Project were also assessed by survey 
questionnaires. 

Questionnaires were developed to ascertain the perceptions of the teachers 
involved in each condition of the Cognitive Educatton Project and were administered on an 
annual basis. The purpose of these questionnaires was to determine the teachers' 
perceptions of the project, including support and consultatk)n provided by the team, test 
administration concerns, usefulness of test data provided, pupil behavioral change, 
inservice effectiveness, appropriateness of experimental program to variatbn in grade, 
class size, and time altotted for strategy instruction. Also, a follow-up teacher 
questionnaire was administered after the completion of the study. 

A questionnaire regarding the principal's perception of both the implementation 
of the experimental programs and participation in the Cognitive Education Project was 
distributed to all principals on an annua! basis. A follow-up survey was also 
administered to principals of experimental condition schools after the completion of the 
project to determine their perceptions regarding their involvement in the study. 
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With respect to parents' perceptions, a questionnaire was sent to alt subjects* 
parents. In this questionnaire parents were asked if they recognized any positive 
changes in their child in the following nine behaviors: attention to homework, tinfie spent 
on task, ability to accept criticism, willingness to tackle more difficult tasks, 
questioning, alternative points of view, self-confidence, originality in thinking, and 
vocabulary. 

Summary of methodology 

The study involved the evaluation of two learning/thinking strategy teaching programs 
in comparison to tradittonal classroom instruction in elementary and junior high 
schools in north-central Alberta. Teachers received inservice training and then taught 
the programs for a period of two years. Strategy instruction was followed by one year of 
maintenance, during which all strategy instruction was withheld. Nine hundred pupils 
comprising gifted, average, and learning disabled students were initially identified to be 
followed. Change with respect to cognitive ability, reading and math achievement, affect, 
perceived competence and cognitive strategies in reading and math was evaluated over the 
course of the three years. As well, the perceptions of teachers, administrators and 
parents with respect to the two experimental programs were assessed. 

Results 

The results of the study are discussed in terms of the effects of the two cognitive 
education programs and conventional instruction on student performance in four major 
areas: cognitive ability, academic achievement, affective perceptions, and cognitive 
strategies. The major analyses were two-way ANOVAs with the first factor being 
experimental group (I.E., S-P.E.LT., Control) and the second a repeated factor being the 
testing periods. 

Grade 4 results 

In the following sections, the results are reported first for the learning disabled, then 
for the average-achieving, and finally for the gifted students. 

Learning disabled 

For the learning disabled students at the grade 4 level, a number of experimental 
program effects were observed, except for the cognitive ability measure. The most 
pervasive experimental effects were observed for academic achievement, 
particularly in the area of reading comprehension. Figure 1 shows that both I.E. and 
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S.P.E.L.T. Students appear to display greater gains in reading compreiiension grade 
scores after two years of cognitive instruction when compared to tfieir control 
counterparts. However, I.E. students appeared to regress during the maintenance period 
performing at essentially the same level as Controls by the end of the three-year period. 
S.P.E.L.T. students on the other hand, demonstrate little change during the maintenance 
period and after three years of instruction are performing at about one grade level 
higher than both I.E. and Control students. 



Figure 1 

Reading Comprehension: Grade 4 Learning Disabled 
£ n 
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There were also program effects observed for math concepts and application. The 
results indicate program impact as both the I.E. and S.P.E.L.T. groups show increased 
performance over the Control group at the end of the maintenance year (see Figure 2). 
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Figure 2 

Math Concepts and Application: Grade 4 learning disabled 
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The standardized reading achievement changes were coupled with changes in related 
strategy areas of reading most notably in metacognitive reading awareness (see 
Figure 3) and to a lesser extent in cloze performance and comprehension monitoring. 
These related effects were more consistently observed for S.P.E.L.T. students. This 
might be expected since many of the reading strategies taught in the S.P.E.LT. program 
were more directly related to reading tasks than are those in the I.E. program. 

Figure 3 

Metacognitive Reading Awareness: Grade 4 Learning Disabled 
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Also, the improvement of math problem solving skills of the I.E. and SP.El.T. 
students was coupled with some changes in the frequency of math strategies used in 
solving individual problems. This was particularly true for the S.P.E.L.T. students who 
displayed a greater frequency of use of the reread strategy as well as the strategy stating 
a plan during problem solution. 

In terms of affective perceptions, there were no significant program effects 
observed with respect to changes in students' perceived competence or self-concept. 
However, experimental effects were observed with respect to Icjus of control The 
S.P.E.LT. and I.E. students displayed an increase in overall internal locus of control after 
the two years of instruction. 

In short, the general effects revealed here provide a reasonably optimistic 
picture regarding the impact of cognitive edi .ation for students with learning 
difficulties in regular classrooms. 

Average-achieving 

For grade 4 average-achieving students, the results indicated that the groups were 
performing at essentially the same level on the cognitive ability measure at both pre and 
post-test. 

With respect to academic achievement, no significant program effects were 
observed. Differences between groups in frequency of different strategy use when 
solving math problems were observed. For example, a significant group main effect, as 
well as a group-by-time interaction, was obtained for the strategy of ''stating plans". 
Figure 4 displays the results. It appears that the interaction is due to the behavior of the 
S.P.E.LT. students. After one year of instruction S.P.E.LT. students displayed a higher 
frequency of use of the strategy as compared to both I.E. and Control students and this 
tended to endure over the three years. There was also a greater overall frequency of use 
of a reread strategy observed for S.P.E.LT. students as compared to both control and I.E. 
students. 
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Figure 4 
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With regard to affective perceptions, little effect was observed with respect to 
self-concept and perceived competence. There were, however, experlmenta! effects 
obtamed with respect to locus of control. After the two years of instruction both 
S.P.E.L.T. and I.E. students displayed a greater degree of internal locus of control than 
students in the traditional program. 

Gifted 

For the grade 4 gifted students, the results indicated some changes in student performance 
in terms of cognitive strategies in the area of reading comprehension and, to some 
extent, in math problem solving. 

Specifically, the reading cloze performance of both I.E. students and S.P.E.L.T. 
students was better than for Controls particularly by the end of maintenance year (see 
Figure 5). This was coupled with improved comprehension monitoring for both 
S.P.E.LT. and I.E. students as well as a trend for increased metacognitive reading 
awareness. Ceiling effects were observed for the standardized reading measures, thus no 
significant increases could be expected with respect to the standardized measure of 
reading. 
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Figure 5 
Cloze (Synonyms): Grade 4 Gifted 
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Figure 6 

Strategy (Determining Alternative Ways): Grade 4 Gifted 
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I.E. and S.P.E.LT. students displayed greater frequency in the use of the strategy 
of determining alternative ways of solving a problem as well as the strategy of 
determining the reasonableness of an answer when cx)nfronted with math word problems. 
For the strategy of determining alternatives, both S.P.E.LT. and I.E. students 
demonstrated greater use of determining alternative ways of solving problems, 
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particularly at the end of maintenance (see Figure 6). This is clearly the case for I.E. 
students who displayed an increasing frequency of use of this strategy over time. 

Regarding affective perceptions, there were no significant program effects 
observed with respect to changes in students' perceived competence, self-concept or 
locus of control. 

Grade 7 results 

The results for the grade 7 students are presented in the same order and format as was 
the case for the grade 4 students. 

Learning disabled 

For the learning disabled students at the grade 7 level, no significant program effects 
were observed for verbal or quantitative scale scores of the cognitive ability measure. 
However, a significant group-by-time interaction was obtained with respect to non- 
verbal ability. The interaction appears to be due to an increase in non-verbal ability 
for the S.P.E.LT. students as compared to I.E. and Control students who perfonned 
essentially at the same level or decreased slightly. The pre-test to post-test means for 
I.E., S.P.E.L.T., and Control were 98.8 to 101.3, 103.2 to 108.3, and 98.5 to 99.8 
respectively. 

With regard to academic achievement, trends toward program effects were 
obsen/ed for reading comprehension and vocabulary, the findings displayed some changes 
as a result of program implementation and this was particularly true for S.P.E.L.T. 
students (see Figures 7 and 8). 
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Reading Comprehension: Grade 7 Learning Disabled 
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Figure 8 



Reading Vocabulary: Grade 7 Learning Disabled 
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Changes were also observed after two years of instruction for standardized 
measures of both math computation and math concepts and application. I.E. students 
performed better in math computation than either S.P.E.LT. or Control students at the 
end of the two years of instruction, but this difference disappeared at the end of 
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maintenance. This pattern for I.E. students was also observed for math concepts and 
application. 

For the measures of affective perceptions, the experimental programs appear to 
have had little effect on the perceived competence, perceived problem solving ability, 
and self-concept of adolescent learning disabled students over a three-year period of 
time. However, the degree of internal locus of control was affected for students in the 
S.P.E.LT. program more so than I.E. or Control students. 

In terms of cognitive strategies, the reading and math related strategy measures 
displayed some changes as a result of program implementation. This was particularly 
true for S.P.E.LT. students. Although the results were not consistently significant, the 
majority of trends indicated changes in favor of the experimental students. 

Average*achieving 

There were few obvious experimental effects observed for grade 7 average achieving 
students. There was some indication of increased reading performance in using contextual 
strategies in reading after the three years (as measured by the cloze task) for S.P.E.LT. 
students as compared to I.E. and Control students. Few other significant effects were 
observed. 

Gifted 

The results for the grade 7 gifted students displayed some interesting patterns. There 
were some changes in achievement in math computation and math concepts and application. 
The standardized math computation performance of the I.E. students, and to a lesser extent 
the S.P.E.LT. students, appeared to be affected after two years of instruction. The gains 
over the two years for I.E., S.P.E.LT,, and Control students were 2.9, 3.0 and 2.6, 
respectively. These differences tended to disappear at maintenance with Controls doing 
better. However, ceiling effects were beginning to play a part at both the end of two years 
and at maintenance. The math concepts and application performance appeared also to be 
affected to some extent by both I.E. and S.P.E.LT. teaching. The gains, after two years of 
instruction for I.E., S.P.E.LT., and Control, were grade equivalents of 2.0, 2.0 and 1.1, 
respectively. 

With respect to reading performance, there were no significant effects observed 
for standardized measures of reading. This is to be expected since the level of performance 
by all of the gifted students was high at the outset. 

There was, however, evidence for positive change in metacognitive reading 
strategy awareness for both I.E. and S.P.E.LT. students. The means for I.E., S.P.E.LT. and 
Control students were 34.6, 34,9 and 31,5, respectively. Overall the two cognitive 
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education programs appeared to significantly affect gifted grade 7 students' 
metacognitive awareness in reading. After approximately six months of instruction, 
the metacognitive reading awareness of these students appeared to be enhanced by both 
experimental programs and this difference was maintained over time (see Figure 9). 



Figure 9 

Metacognitive Reading Awareness: Grade 7 Gifted 
36-1 m 
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The math strategies students chose to use in solving particular math problems 
also demonstrated some experimental effects particularly for I.E. students and to a lesser 
extent S.P.E.L.T. students. For example, a significant group-by-time interaction was 
obtained for the frequency of usage of the rereading strategy. At the post year 2 
point, white Control and I.E. increased, the S.P.E.LT. score remained relatively stable. 
At maintenance, the interaction was due to increasing I.E. scores compared to decreasing 
S.P.E.L.T. and Control scores. The I.E. students displayed greater usage of the rereading 
strategy as compared to both Control and S.P.E.LT. students after maintenance (see 
Figure 10). 
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Figure 10 

Math Strategy (Rereading): Grade 7 Gifted 
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Finally, with respect to affective perceptions, no significant program effects 
were observed for any of the three measures of affective perceptions. 



Participants' Perceptions 

In order to monitor and evaluate the evolving perceptions of each of the participating 
groups (teachers, principals and parents), with respect to their involvement in the 
Cognitive Education Project, survey questionnaires were developed and administered to 
each group on an annual basis for three years. Presented here are summaries of the 
information gathered from these surveys. 

Teachers' perceptions 

The perceptions of the participating teachers were extremely positive for both I.E. and 
S.P.E.LT. Immediately at the end of implementation and project involvement, all 
teachers in S.P.E.L.T. and the majority of I.E. teachers indicated that they would like to 
continue using the procedures. When asked one and a half to two years later, a 
significant number of teachers (85%) indicated they were indeed teaching certain 
aspects of the program to their classes. All the teachers in I.E. and S.P.E.LT. indicated 
that they would recommend their respective programs to their colleagues and 
furthermore, that involvement in the training had enhanced their professional 
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development. The inservice training and follow-up support were also cx)nsidered to be 
appropriate. In addition, some teachers indicated that more inservice days spread over 
the year after the initial three-day inservice would have been more beneficial. Teachers 
generally felt that both experimental programs were effective for all student ability 
levels, although some I.E. junior high teachers indicated that the program was most 
appropriate for low ability students. 

Principals* perceptions 

Very few differences were noted between the questionnaire responses given by the 
S.P.E.L.T. and the I.E. principals. Generally, principals involved in the experimental 
conditions viewed their teachers as being relatively enthusiastic about the project both 
at the conclusion of the inservice and over the course of the year. 

Problems were noted for scheduling and implementation in the I.E. condition. 
While I.E. principals encountered difficulties in the scheduling and implementing of the 
required three forty-minute classes outside of the regular curriculum, this was not the 
case for S.P.E.L.T. principals. I.E. principals reported both personal and teacher 
frustration arising from these timetabling difficulties. Such problems apparently 
affected grades 7 and 8 more frequently than the grades 4 and 5 classes. 

Parents' perceptions 

The perceptions of parents with respect to changes in their children suggested that there 
were generalizing effects being obsen/ed at home. A number of behavioral changes were 
related and were consistently being reported more often by I.E. and S.P.E.L.T. parents 
than Controls. The most frequently reported behavioral changes over grades were in 
attention to homework, in recognizing alternative points of view, in 
willingness to tackle more difficult tasks, in self-confidence, and in 
questioning. The parent data strongly support the idea that program effects are being 
generalized for both S.P.E.L.T. and I.E. students, particularly after one year of 
instruction and at the end of maintenance. 
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Summary of Results 

student change 

The results appear very promising, particularly for learning disabled students and to a 
sonfiewhat lesser extent gifted students (see Table 2). The most pronounced effects were 
observed for the grade 4 learning disabled students, most notably in reading 
comprehension and related strategies. There was also evidence that the average and 
gifted students benefited, but to a lesser degree. Generally, S.P.E.L.T instruction tended 
to produce more changes as compared to I.E. and Control. This finding was not unexpected 
since the S.P.E.LT. instruction, in large part, involves teaching cognitive strategies 
directly within curricular content areas. 

The lack of consistent maintenance of behavioral change, which was sometimes 
observed for I.E. students, may be due to insufficient time allotted for I.E. instruction. 
Maintenance of the program might well be achieved if I.E. Instruction could have been 
continued for a longer period of time. 

This study is a highly conservative one. as in nfiost cases only one or two teachers 
in a school, at one grade and subject level, taught either the I.E. or ScP.E.L.T. program. 
If all teachers at each grade level were engaged in the teaching of cognitive education 
procedures, quicker and more comprehensive effects would likely have emerged. There 
is some evidence for this expectation in a study conducted in Barrhead and Swan Hills 
schools in 1985 (Mulcahy, Peat, & Darko-Yeboah, 1986). All teachers from grades 
4 to 12 were trained in cognitive education procedures, with pre- and post-tests 
administered to students during the initial year. Significant pre-post gains were noted 
on measures of self-concept, perceived problem solving, metacognitive reading 
awareness, and strategy use. 

The results obtained in the present study are also consistent with findings from 
recent research on the teaching of metacognitive strategies (Paris & Oka, 1986). 
Haller, Child and Walberg (1988) conducted a meta-analysis of twenty studies of 
metacognitive strategy teaching with respect to reading comprehension which involved a 
total of 1 ,553 students. They obtained an average effect size of .71 which they report as 
one of the largest uncovered in educational research to date. This supports the claim that 
metacognitive strategy teaching is effective in making children more aware of reading 
strategy variables. 
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Table 2 

Summary Chart of the Three-Year Results 
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Reading Cloze Performance 
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Participants* perceptions 

The perceptions of parents, teachers and administrators regarding the two cognitive 
education programs are extremely encouraging. The vast majority of experimental 
teachers reported that they would continue to use the instructional procedures from the 
two programs. Indeed when a significant number was surveyed again in November 
1988, two years after involvement in the project, over 85% of the teachers reported 
they were continuing to use aspects of the programs in their teaching. This was coupled 
with the fact that all teachers indicated they would still recommend the programs to 
their colleagues. 

Parents* responses, although limited to the second and third years, also indicated 
positive changes in their youngsters, in a number of important behavioral areas such as 
self-confidence, task persistence, accepting alternative points of view, 
originality of thinlcing, and questioning. 

Administrators* perceptions of the two experimental programs were also 
generally very positive. SCwte concern was evident with respect to the high cost of I.E. 
materials as well as the practice of teaching I.E. in isolation from the rest of the 
curriculum. However, on follow-up questionnaires, one and one-half to two years after 
their involvement in the initial implementation, over half of the 24 principals said they 
would consider adopting the experimental programs. Many of these principals stated 
they would also recommend the cognitive education programs to other schools. 

inservice training 

With respect to the inservice training provided for the two experimental programs, the 
vast majority of the teachers indicated that inservice training was sufficient to allow 
them to implement the programs adequately. There was some indication from a numlDer 
of teachers that more **in-school support** would have enhanced implementation of the 
programs. A feeling of isolation (being the only teacher in the school involved), was 
voiced on occasion by some teachers. The administrators and other staff were not 
familiar with the programs. Nevertheless, teachers were able to learn and implement 
the programs with some degree of facility. The need to have all staff become familiar and 
involved from the beginning of the program was evident. 

The teachers in both experimental programs indicated the desirability for 
additional follow-up inservice sessions over the years. This could be handled to a 
considerable extent through peer coaching and staff meetings devoted to discussion of 
teaching procedures, and generalization over content areas and grades. 
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Implications 

Cognitive education as a part of school curriculum 

The results observed with respect to pupil change, coupled with the perceptbns of 
parents, teachers and administrators, suggest that the teaching of learning/thinking 
strategies should be made an integral part of the school curriculum. 

The question of whether the cognitive education programs are more appropriate 
at different grade levels or for different diagnostic groups could not be answered 
definitively. However, instruction at the lower grades was associated with better gains 
than instruction at the higher grades. The results clearly indicate that both programs 
have a greater effect on students at the grade 4 level. The teachers involved in the 
experimental conditions also indicated general appropriateness of both programs for 
grade 4 students. Both programs appear to be most effective for grades 4 and 7 learning 
disabled students and to a lesser extent for gifted students. It is somewhat puzzling that 
there appeared to be less Impact on average students as compared to learning disabled or 
gifted students. This may be due to the fact that learning disabled students generally lack 
a systematic strategic approach to tasks and thus benefit more quickly when provided 
with a systematic approach. Average students may already have a somewhat effective 
approach in place, and thus fail to benefit significantly from the programs at the outset. 
Gifted students have the Intellectual ability to perceive the usefulness of the strategies 
and then to use and extend them immediately. Many of the teachers commented that they 
found the higher ability students to "take off with the strategies in extending and 
applying them. 

Cognitive education for students 

The results have clear implications for the mainstreaming of students with learning 
difftoulties, as well as gifted students. The Impact of the teaching of cognitive strategies 
on the learning disabled students, particularly at grade 4, suggests that if the teaching 
approaches are used systematically throughout the elementary school, it may prevent 
some students from developing severe learning problems, and keep them in the 
mainstream. 

The recent research on the teaching of learning/thinking strategies to learning 
disabled students also demonstrates significant effect with respect to achievement (see 
for instance Palincsar & Brown, 1984; Paris & Oka, 1989; Goldman, 1989). The 
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effects observed in this study for gifted students also indicate that they may benefit from 
such teaching in the regular classroom. 

These approaches should also be effective for mildly mentally retarded as well as 
native youngsters in regular classrooms. Indeed some researchers have suggested this 
(Mulcahy & Marfo, 1987; Brown, 1984). There is a need for further research on 
teaching learning/thinking strategies for these populations. Further investigation into 
the effects of extending cognitive instructbn to primary, senior high and post-secondary 
levels is also required. Current research at the preschool level with high risk children, 
appears to hold significant promise (Haywood, Brooks, & Burns, 1986; Price, 1991). 

Assessment of cognitive strategies 

The two cognitive education programs affected the cognitive strategies utilized by 
students after instruction as well as Increased their degree of metacognitive reading 
awareness. The comprehension monitoring skills of students after two years of 
instruction in the two cognitive education programs were observed to be generally better 
than their control counterparts, and so was their performance on a cloze reading task. 
Both these tasks require students to use context to fill in missing words and to determine 
the comprehensibility of the text. As Paris and Oka (1986) have Sv-rjgested, skimming, 
rereading, paraphrasing, inferring and checking are strategies that one wouki use on 
these tasks. No direct measures of these particular strategies were taken during the 
Cognitive Education Project. However, the changes in math strategies noted in I.E. and 
S.P.E.LT. students were often those of rereading, stating plans and determining 
alternative solutions. 

The assessment of cognitive strategies was addressed to some extent by the 
project. However, it may be that more pervasive strategic change could have been 
identified if more appropriate criterion measures of cognitive strategies were available. 
Further research must address the development of these instruments for researcher as 
well as practitioner use. The math problem solving strategy approach, the Metacognitive 
Reading Strategy Awareness Inventory, and to some extent, the Perceived Problem 
Solving Inventory, used in this study appear to hold some promise both from face validity 
and/or from results reported here and elsewhere. 
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Reflections of the Research Team 

The study reported here is a very extensive one involving an extremely large number of 
participants. When one is involved with such a large number of individuals over an 
extended period of time, there is a great deal of what might be termed qualitative data 
collected which can shed further light on the study about cautions which future studies of 
this nature can benefit from. The following provides some of this qualitative data from the 
perspective of the research team. There are essentially six major topics of importance 
related to the study which we feel merit attention: data interpretation, inservice training, 
implementation, program management, evaluation, and programs. 

Some considerations regarding interpretation 

As with any study there are significant limitations which must be considered in 
interpreting the results reported. Among the more critical with regard to this study 
are: with the large number of statistical analyses significant results by chan'^e are 
increased; specific tests of program mastery were not conducted independently by the 
research team so it is possible that all students may not have mastered the content of the 
experimental programs; the majority of situations involved only one experimental 
teacher in a school thus restricting the instruction to one classroom of students as well 
as the consolidation and generalization of the learning and thinking strategies; the small 
number of parent questionnaire returns in years two and three restricts the degree of 
confidence that can be placed on these results, however, the results at the end of one year 
of instruction can be viewed with confidence in light of relatively high rates of return; 
not all students had all measures available for analysis, however, missing data due to 
attrition, test administration, etc., is a significant problem and is particularly a 
difficulty in longitudinal studies; the use of grade equivalent scores as the dependent 
measures in some of the analyses could be criticized. 

Inservice training 

Due to the large number of requests for Inservice training in S.P.E.L.T., there has been 
much opportunity to modify and refine the S.P.E.L.T. inservice model. As well, through 
classroom observations throughout the four years of the Cognitive Education Project, 
insights were gained as to which aspects of inservice training were applied most 
frequently at the classroom level. The synthesis of this information is presented below. 

S.P.E.L.T.'s three-phase approach to the teaching of learning/thinking strategies 
appears to be more important than the choice or sequence of strategies used. This 
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observation is strengthened by recent research articles which emphasize 'informed 
strategy training' (for example, see Nickerson, 1988). As well, there was a tendency 
for some teachers in the Cognitive Education Project S.P.E.LT. condition to remain 
within the direct teaching of Phase I for extended periods of time, rather than moving to 
Phases II and III. In light of these factors, and based upon the responses of hundreds of 
teachers to the inservice training process, the inservice training model has been 
modified as follows: 

a) Rather than spending each of the three days of training on each phase of the 
S.P.E.LT. instructional model sequentially (i.e., Day 1«Phase I; Day 2=Phase II; Day 
3«Phase III), the strategies are now presented in content-based clusters, with each 
cluster moving through the three phases. This results in teachers undergoing an 
experience with the S.P.E.LT. instructional continuum a minimum of four times 
throughout the three-day training process. Although the number of strategies presented 
is reduced using this model, the importance of the instructional continuum is 
enrtphasized. 

b) The emphasis on the instructional continuum, with multiple strategy clusters 
being presented, rather than the individual strategies, seems to eliminate the tendency 
of some teachers to emphasize only one area of strategy application such as reading 
comprehension or memory. In other words, the breadth of strategies used in classrooms 
appears to be greater than when the original inservice training model was used. 

The five-day inservices for I.E. should be split into two- and three-day blocks in 
order for teachers to try the principles of I.E. before completing the entire training. In 
addition, two half-day workshops held at regular intervals would give teachers a chance 
to share problems and ideas and to renew their enthusiasm with respect to the program. 

In order to ensure that teachers are able to acquire the necessary teaching skills 
and strategies regardless of the program, cognitive coaching should be made an integral 
part of the initial training. This coaching for application would Involve hands-on, in- 
classroom assistance and cognitive modelling to transfer skills and strategies to the 
classroom. This could be achieved through the extended training of classroom 
consultants who could then work with teachers in the classroom on a continual basis 
after the initial inservice training. Peer coaching should also be made an integral part 
of the inservice training and extended into the classroom situation. 
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Implementation 

The question of whether the programs were implemented as intended, was determined 
through on-site visits and phone calls. In the majority of cases the programs appeared 
to have been implemented adequately; however, it is clear that some teachers were 
implementing to a greater degree of intensity than others. Teacher enthusiasm with 
respect to the two particular approaches was extremely high, so it was difficult to 
control for the Hawthorne effect. However as has been pointed out by other researchers, 
it is unrealistic to expect these teachers to hide their enthusiasm. Moreover, teacher 
enthusiasm is useful evaluation data (Nickerson, 1986) indicating face validity of the 
programs. 

It is also possible in the present study that the control group's performance was 
enhanced as a result of participation in the project. There were some indications from 
control teachers to suggest this. One particular comment voiced to the principal 
researcher was, "I bet my students will do as good or better than those students being 
taught strategies-, suggesting that some control teachers may have viewed themselves in 
competition with the experimental groups. Even with the careful monitoring it is still 
possible that programs were not as systematically implemented as intended. In order to 
ensure maximum implementation it would be helpful to have peer teacher coaching made 
an integral part of the procedure. The first year could be a program implementation 
year and the following years, evaluation years. 

When future studies of this nature are considered, it would be more appropriate 
to have the initial year as an implementation year where teachers are trained and 
monitored through the year and the coaching is developed as an integral part of the 
program. The succeeding years would then become evaluation years which would result 
in less confusion and more direct evaluation of program effects. 

When inservice training pre-testing and implementation are all occurring at the 
same time, teachers, students and administrators as well as the research team are 
dividing attention and efforts towards many variables and as a result one or more of these 
areas suffers. If participants are able to concentrate on one aspect at a time, it is likely 
that more appropriate implementation might occur. Allowing the initial year to enable 
teachers to concentrate on implementing the procedure would result in less confusion. 
This would also enable the research team to engage in more in-class coaching and 
monitoring to assist in more appropriate implementation. This raises another problem 
that we ran into in the implementation which was a major concern for the teachers as 
well as for the research team. The feedback from a number of teachers in the field 
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indicated that they felt they were being torn in a number of directions simultaneously and 
thus were unable to put forth a concentrated effort to the implementation. One issue was 
that many districts had a particular focus one year with respect to the teaching and 
enhancing effectiveness and the next, a new thrust or focus would emerge. An example 
might be the emphasis on teacher effectiveness training and then next year focus on 
increasing students* self-esteem. What this tended to do was restrict teachers in the 
ability to implement the strategy teaching consistently over the years with a ft>cused 
thrust. The comment was often heard, Ve are being asked to do too many things at the 
same time with not a long enough period with any one focus to enable us to do the task 
adequately". The research is clear that the teaching of learning/thinking strategies Is not 
a brief task but one that requires a k>ng-term commitment over a number of years. 

The experience gained through implementing this project pointed out a number of 
pitfalls that need to be avoided when schools or school systems contemplate putting in 
place a program of learning/thinking strategy instruction. Some of these have also been 
documented by Chance (1986). 

There was often a tendency to move too fast or try to do too much in a short period 
of time. It was evident that in many instances too many cognitive strategies were 
attempted to be taught in a very short time. It is important to focus on a few strategies 
over the course of a year or term and teach them well. 

It appeared In many cases that expectations for change were too high. The 
expectation of signifk:ant changes in a short period of time could have led to some 
disenchantment and the abandonment of the teaching or at the most a halfhearted effort. 
Cognitive strategy instruction is a long-term commitment and must be viewed as such. 

Giving in to early failures can be a common problem. Instead of viewing these as 
failures, it is important to treat them as opportunities to learn more about cognitive 
strategy teaching. In these early stages of the development of cognitive instruction, 
mistakes and failures are to be expected and can be the vehicles for adding to our 
knowledge regarding how best to deliver cognitive educatk>n instruction. Just as we 
encourage students to use errors in their problem solving to assist them in their 
learning, so too should teachers of cognitive education. 

Progrann management 

Record keeping, collecting and scoring tests, and computerizing results were onerous 
tasks. The recording of what decisions had been made and why with regard to testing, 
inservice, questionnaire distributions, scoring, student attrition, assessment 
diffkujlties, and missing data to list but a few is a full-time job. It is necessary to keep 
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very careful written records of dates, times, decisions and individuals involved in order 
to be able to interpret study data in a realistic way. This is something that we in the 
study team did as consistently as possible. However, there were still times it was not 
totally dear what had happened and why. 

Communication in a project of this nature is one of the key elements which can 
make or break the study. It is essential that all major participants are kept informed 
from beginning to erKl with respect to all aspects of the study. With a study of this 
magnitude, this is an extremely difficult task. At the very beginning in order to make 
explicit as possible each participant's role, a summary of the project was written up 
including questions to be addressed ak>ng with methodology. The particular roles and 
responsibilities of the school jurisdictk>n, teachers, and research team were clearly 
defined in this summary which was then provided to participants prior to involvement. 
A signed written contract from the participating school jurisdicttons was then requested 
to ensure to the degree possible, the continued commitment for the three-year duration 
of the study. As well, some meetings were held with school boards, parents, and/or 
administrators to communtoate the study's intent. In future studies, meetings of this 
nature should be an integral part of initial pr-* :ram management to ensure 
communication is appropriate and to address any questtons or concerns. 

Over the course of the study it is essential to ensure ongoing communication 
between the research team and field. We attempted to do this by requesting one contact 
person (usually an assistant or associate superintendent) to be assigned to communicate 
with the research team and teachers. In addition phone calls and newsletters were used to 
maintain contact with schools up to 80 to 100 miles away. In future, it may be useful to 
use a computer networking system with the schools and set up electronic bulletin boards 
where sharing of infomnation could occur between the team and the fieki as well as 
between teachers in the field involved in the same program. This would alk)w a sharing 
of ideas with respect to program implementatton as well as helping to maintain a focus 
on the implementation. 

Following a small number of children out of a total classroom is extremely 
difficult, particularly over more than one year. Testing becomes disruptive to the 
entire class and teachers are unable to conduct group testing easily as they have to divide 
their time between those students being followed and the rest of the class. 

When following these students into the next grade» significant difficulties were 
encountered with respect to both students and teacher. Typically students are reassigned 
to classrooms at the end of each year which means that a new group of students is formed 
each year. Thus some of the students had been taught the program for one year whereas 



ERIC 



43 



33 



Others had no knowledge of the program. In the project we attempted to have students 
move as a complete group to the next class, however, this was only achieved in a very 
small number of cases at grade 4. If this was not possible then those students who were 
being followed were kept together as a group in their new class. The teacher then 
reviewed the previous year's work emphasizing those aspects crucial to the program 
nrK)vement to the next level. Identifying teachers at the beginning who woukj become 
involved in the second year and training them In the initial year would be extremely 
helpful in having teachers ready to teach the next level of the programs. If all teachers 
were involved this would not be as much of a problem. 

All of the testing would best be done by the research team to ensure consistency 
as well as greater validity and reliability of data. Teachers might best administer only 
those tests directly usable and interpretable by them. 

Evaluation 

In studies of this nature, the question of the most appropriate criterion measures is a 
major concem. We attempted to evaluate the program from a number of perspectives 
including the cognitive strategies employed by the students. There is a need in the 
continuing work of this nature to further develop instruments and techniques more 
sensitive to the development of learning/thinking strategies and their transfer. 

In evaluating the implementation it is obvious to us that we have not been able to 
detect changes that may have occurred. Teachers would often comment that the 
implementation of the programs had positively changed their teaching style and the way 
they now viewed and interacted with their students. Many teachers teaching 
Instrumental Enrichment for example, indicated when visited in their classrooms that 
students who previously would not volunteer an answer in class or debate a point would 
now do so, thus increasing their confidence in their work. It would be extremely 
beneficial in future research to attempt to objectively document these reported changes 
both through classroom observation as well as test-retest data. This was not directly 
observable from our measures. A classroom interaction scale, evaluating those student- 
teacher as well as student-student behaviors one would expect should change as a result 
of the strategy instruction, would provide an important evaluation component which has 
been missing from inten/ention studies in the teaching of cognitive strategies. 

In further studies attempt should be made to determine differential impact with 
respect to quality and degree of program implementation, as it appeared obvious that 
some teachers were clearly implementing with greater enthusiasm and commitment than 
others. This might be built into the initial design as a major factor to be evaluated. 
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As this study progressed we became aware of new instruments which may in 
future be excellent criterion measures of program impact. One such measure is the 
Learning Process Questionnaire (LPQ) (Biggs, 1985). This particular instrument 
evaluates students* learning motives and strategies in an interactive way and thus may 
be a very good measure of changes in motivation and strategy as a result of program 
implementation. The Structure of the Observed Learning Outcome (SOLO) (Biggs & 
Collis, 1982) method of determining students' levels of thinking in different content 
areas also appears to warrant consideration as an approach for evaluating students' 
learning with respect to the teaching of learning/thinking strategies. These measures 
should be explored along with others in future studies of this type. 

One acceptable measure of program impact is the degree of interest generated as 
the project progresses and the willingness of other groups and individuals to commit 
resources and time to the program. There was a significant increase in interest 
exhibited locally, nationally and internationally by teachers, administrators, and 
researchers which has continued to grow. It is obvious that the study has now 
progressed to the diffusion stage where the teaching program (particularly S.P.E.L.T.) 
is now being incorporated at a variety of levels in different classrooms, school systems 
and universities. 

Programs 

The experience gained from working intensively with the two different programs was 
invaluable. The difficulties we encountered with implementing a program which takes 
time away from the content were many and varied. One major difficulty was with the 
conflict teachers felt with respect to covering the curriculum in order not to detract 
from their students' performance on the tests required by Alberta Education. The felt 
need to cover the curriculum content was so great that in many instances, it was 
obvk)usly detracting from the implementatton of the program in terms of time altocated 
as well as focus. This was particularly a problem with respect to Instrumental 
Enrichment but was also voiced by some teachers with respect to S.P.E.L.T. 

Both the research team as well as teachers implementing the two programs 
observed that it may well be that the principles of the Instrumental Enrichment 
program might be utilized effectively with some students displaying particular cognitive 
deficits at initial stages and then the S.P.E.L.T. procedures coukJ be brought in to further 
extend the principles developed through I.E. There might thus be a blending of both 
programs with S.P.E.L.T. assisting extensively in the bridging and extending of strategies 
to content areas. 
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Some teachers indicated that they would like to see the principles of teaching 
found in Instrumental Enrichment to be more freely allowed to be adapted or extended 
beyond the actual exercises and that the more important component was the mediated 
style of teaching. The program might well benefit from further development in a 
broader context. 

A large percentaqe of teachers voiced their concern that their students were not 
being tested v^ith respect to learning/thinking skills but rather on acquired knowledge of 
content and thus teaching learning/thinking skills had to be secondary to teaching factual 
knowledge with respect to curriculum content. If teaching learning/thinking skills is to 
become an integral and focused part of classroom teaching, then assessment of the 
acquisition and application of learning/thinking strategies nnust become a part of the 
evaluation of curriculum in schools. Until it is obvious that this is the case, the teaching 
of learning/thinking strategies will take a backseat to attempting to cram as much 
factual information as possible in order to answer content-based questions, thus 
contributing to the futile acquisition of "inert knowledge" which is adequate for the test 
but fails to be applied or evaluated beyond the test situation. This is not to suggest that 
teaching content is not important but rather a more prominent role will need to be given 
with respect to teaching the process of learning/thinking. 
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Recommendations 

Based upon the results of this study the following recommendations are made. 
Alberta Education 

1 . It is recommended that Alberta Education make the teaching of cognitive education 
procedures an Integral part of the Alberta school curriculum for elementary and 
junior high students and, that this become policy. The vast majority of experts in 
the field indicate this should be integrated into content teaching. The results of 
this study would support this. 

2. It is recommended that Alberta Education develop and make available appropriate 
resource materials for teachers to use in the teaching of cognitive education 
procedures. 

3 . It is recommended that Alberta Education develop and make available to teachers 
and school administrators cognitive strategy assessment instruments in differing 
content areas as well as affective domains. These should be appropriate for both 
formative and summative evaluation purposes. This assessment might be made as 
part of Alberta Education's ongoing program of testing for school jurisdictions. 

4 . It is recommended that Alberta Education make available to all its field consultants 
training opportunities in theory, research, and application issues relating to 
cognitive education procedures. 

Universities 

1 . It is recommended that teacher training programs in Alberta provide compulsory 
training in the principles and practice of cognitive education for ali preservice 
teachers. 

School jurisdictions 

1 . It is recommended that inservice training be made available with respect to 
methodologies for teaching and evaluating learning/thinking strategies in 
classrooms. 
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Joint university, Alberta Education and school jurisdictions 

1 . it is recommended that a Cognitive Science Unit be established which would have 
the following as its major functions: 

( 1 ) the tnservice training of teachers; 

( t i ) assisting in the development and implementation of preservice training 

courses at the university level; 
( i t i ) providing consultation to school systems regarding implementation and 

evaluation procedures; 
( i V ) conducting basic and field research on the teaching and assessment of 

cognitive education procedures. 
The setting up of this unit should initially be a joint endeavor between Alberta 
Universities, Alberta Education, and school jurisdictions. 

2. It is recommended that additional research be conducted regarding the efficacy of 
cognitive education procedures at primary and secondary levels as well as with 
different populations, particularly the mildly retarded and native students. There 
is at present little systematic research on the effectiveness of these approaches for 
these students in mainstream classrooms. 

Concluding Remarks 

Education hais tended to easily grab on to new instructional approaches, the most recent 
one being learning/thinking strategy teaching or in somewhat more popular terms 
**metacognitive instruction**. This type of instruction, regardless of which particular 
program is used, attempts to teach students to plan, implement, monitor and evaluate 
specific strategic approaches to tasks. The recent literature, including this study, 
suggests that this type of instruction has the potential to make enduring positive changes 
with respect to student learning and problem solving (see for instance Haller, Child, & 
Walberg, 1988; Paris, & Oka, 1986, 1989; and Palincsar, & Brown, 1987). It is 
likely that no one particular, added-on program, at one instructional level, will provide 
the adequate emphasis needed. In fact, some experts in the field suggest that it may not 
be what you teach (in terms of particular strategies or materials used) but how you 
teach it that is most critical to positive student change (Nickerson, 1988). Many 
teachers in the study reported here would also support this view. 

The results of this study are very encouraging. There are potential benefits to 
students and teachers of implementing cognitive education procedures in mainstream 
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elementary and junior high school classrooms. The study reported here suggests that a 
number of positive changes in student behaviors do occur for different types of learners. 
The approaches examined in this study, however, are only two out of a wide variety 
available and these two might best be viewed as initial attempts at teaching learning/ 
thinking which do hold some promise. Further development and evaluation is stiil 
necessary. 

There are numerous questions still to be answered within the context of this 
particular study including developmental and individual differences of the different 
groups of students with respect to strategy and affect. As well, questions regarding more 
specific subgroup differences with respect to program impact are still left to be 
answered (e.g., those high in self-concept versus low, those high in external locus of 
control versus low, etc.). As well, examining subgroups of students who appeared to 
benefit to a greater degree than others and then attempting to determine why this is the 
case would provide additional insight into differential effects of program impact. These 
and other analyses would help to provide a more comprehensive picture leading to a 
better understanding of the specifics involved in this type of intervention. There are also 
many questions yet to be answered in future work including: How quickly should one 
introduce a new strategy? What is the most powerful way to obtain transfer and 
generalization? What is the best way to interface content and strategy? What is the 
most appropriate way to integrate affect and cognitive strategy teaching? 

Despite the problems and unanswered questions we need to pursue metacognitive 
instruction in our classrooms. There is clearly the need to provide for a comprehensive 
integrated approach to the teaching of cognitive strategies across all levels of education 
beginning at kindergarten through to post-secondary. The most appropriate ways of 
doing this to enhance leaming/thinking have yet to be determined. However, we do now 
know enough to begin to make a start. 
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